Active learning for constrained regression using kernel beta regression models

In this poster we study active learning for supervised regression algorithms. We focus on a particular problem where the regression is constrained to an interval. This is the case, for instance, when the objective is to estimate the probability of success of a certain event given a set of input points. The regression problem is, thus, constrained to the zero-one interval and the training examples are also discrete, e.g. success or failure. The constrained regression problem is usually solved using logistic regression [1]. This is in essence a generalized linear model for discrete outputs (e.g. classes) based on the logits of the probabilities. Active strategies for this type of techniques have been studied in [2] and pointed out a trade off between computational cost for experimental design based techniques and robustness issues that appear in more heuristic criteria. A support vector machine version of the logistic regression, the kernel logistic regression [3] uses the log-likelihood of the binomial distribution as the loss function and provides directly estimates of the probability. Constrained regression has also been studied using Beta models in economics [4], to model rates and proportions [5] and in psychological studies to account for skew and heteroscedasticity [6]. In this case, a parametric model is fitted maximizing the likelihood function using, for instance, Newton-Raphson or Fisher scoring. Better experimental results have been reported using alternative residuals in [7]. Bayesian versions of this regressors have been recently proposed based on a hierarchical model and priors on the parameters [8]. This poster proposes a new algorithm for such a constrained regression problem specially suited for the active learning framework. As logisitc regression, the algorithm uses a Binomial likelihood model. At each point of the input space, a conjugate Binomial-Beta model provides the distribution of the probability of success. The two parameters of the Beta at a particular point x∗ are computed by accumulating evidence of successful and failed events at training points xi using a kernel K(x∗,xi) The expression for the Beta distribution is p(p∗ | x∗,Xn,Yn) ∝ Q n i=0 Bin(S∗i; p∗, S∗i + U∗i)Be(p∗;α0, β0) = Be p∗; n